Bootstrap Variable-selection and Conndence Sets

This paper analyzes estimation by bootstrap variable-selection in a simple Gaussian model where the dimension of the unknown parameter may exceed that of the data. A naive use of the bootstrap in this problem produces risk estimators for candidate variable-selections that have a strong upward bias. Resampling from a less overrtted model removes the bias and leads to bootstrap variable-selections that minimize risk asymptotically. A related bootstrap technique generates conndence sets that are centered at the best bootstrap variable-selection and have two further properties: the asymptotic coverage probability for the unknown parameter is as desired; and the con-dence set is geometrically smaller than a classical competitor. The results suggest a possible approach to conndence sets in other inverse problems where a regularization technique is used.